Efficient Exploration in Resource-Restricted Reinforcement Learning

In many real-world applications of reinforcement learning (RL), performing actions requires consuming certain types resources that are non-replenishable in each episode. Typical include robotic control with limited energy and video games consumable items. tasks resources, we observe popular RL methods such as soft actor critic suffer from poor sample efficiency. The major reason is that, they tend to exhaust fast thus the subsequent exploration severely restricted due absence resources. To address this challenge, first formalize aforementioned problem a resource-restricted learning, then propose novel resource-aware bonus (RAEB) make reasonable usage An appealing feature RAEB it can significantly reduce unnecessary resource-consuming trials while effectively encouraging agent explore unvisited states. Experiments demonstrate proposed outperforms state-of-the-art strategies environments, improving efficiency by up an order magnitude.